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Background / Context: 

International reports on youths conducted by the World Bank (2007) report that civic 
engagement is one of the most important markers of adult transition, alongside school 
completion, attainment of health, employment, and family fonnation (Kassmir & Flanagan, 
2010). In urban areas where ethnic minorities typically reside, there are often few institutions, 
aside from schools that provide them with opportunities to build and participate in communities 
with others (Hart, Atkins & Ford, 2010; Kirshner, Strobel & Fernandez, 2003). Past research 
indicate that civics interventions supplemented in social studies courses can enhance students’ 
civic engagement, as indicated by interest in voting (Syvertsen et ah, 2009), volunteering (Root, 
Northup, & Turnball, 2007), and engaging political discussions with parents (McDevitt & 
Chaffee, 2002). Although past research suggest that civic education programs can be effective in 
helping students become more civic engaged, this study tests whether these treatment effects are 
generalizable across an urban school district with multiple middle school sites. This study also 
contributes understanding of how treatment effects may vary for students from ethnic minority 
populations, in particular to those identified as Asians and Latino. 

cINSERT FIGURE 1> 

The theoretical framework that is tested in this study is the Structure-Stimulus-Orientation- 
Response model (S-S-O-R) model that hypothesizes the relationship between classroom 
discussion of controversial issues and civic engagement (McLeod, Scheufele, & Moy, 1999; 
Scheufele, Nisbet, Brossard, & Nisbet, 2003) In the model: (1) Schools represent heterogeneous 
networks meaning that adolescents have opportunities to interact with others who are unlike 
themselves because they may hold differing beliefs and opinions as opposed to homogenous 
networks (e.g. family, faith, peer-based networks). (2) The stimulus is adolescents’ experience in 
discussing controversial issues in the classroom. Adolescents are exposed to other peoples’ point 
of views. (3) The orientation that develops from the stimulus is Knowledge- Students develop a 
knowledge base in regards to understanding how these issue may pertain to their communities. 
Students may also increase their news media access (going online, reading the newspaper and 
watching television) to understand current events. (4) To address community problems, the 
response is civic engagement- which is participating in certain collective activities such as 
volunteering, discussing issues with peers outside of class and extracurricular activities. 

Purpose / Objective / Research Question / Focus of Study: 

The purpose of this study is to determine whether a cross-content intervention called Word 
Generation has a significant impact on students’ self-reported civic engagement. Secondary 
analysis of this study consists of examining if treatment effects vary across students from ethnic 
minority backgrounds. 

RQ 1: Does the Word Generation Program have a positive impact on students ’ self-reported 
civic engagement? 

RQ 2: To what extent does students ’ ethnic identification (e.g. Latino and Asian) influence 
the direction and strength of the relationship between program participation and self- 
reported civic engagement? 
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Setting: 

A school district located in a West Coast metropolitan area of the United States participated in an 
evaluation study of the Word Generation (WG) program. Once district leaders had been 
recruited, they invited their school level leadership teams to participate in the study. To be 
considered, teams had to accept the prospect of being randomly assigned either to implement the 
program the following fall (“phase 1 schools”) or only after two years (“phase 2 schools”). 

Population / Participants / Subjects: 

cINSERT TABLE 1 > 

From Fall 2011 to Spring 2012, middle school students (N= 5,870) participated in a randomized 
study of the WG intervention that was conducted in an urban school district located in a West 
Coast metropolitan area of the United States. Although an equal number of schools were 
randomly assigned to treatment ( n = 6) and control (n = 6) conditions, the sample contains more 
students who enrolled in treatment ( n = 3,518) than control (n = 2,352) schools. Gender of the 
participants was approximately equally divided with males (50.30%) and females (49.70%). The 
sample consists of students from extremely diverse racial/ethnic backgrounds, which includes: 
Asian (54.1%), Hispanic (19.3%), White (8.6%), and African American (6.9%). As is typical in 
many American urban settings, a majority of the participants (63.8%) are from low SES homes 
(as indicated by eligibility status for the Free and Reduced Lunch program) and 56.8% are 
English language learners. 

Intervention / Program / Practice: 

The Word Generation (WG) is a cross-content program delivered at the classroom or grade level 
that instructs students to learn five all-purpose academic words, which are embedded in brief 
passages covering a different controversial issue each week (Snow, Lawrence, & White, 2009). 
The program features controversial topics that range from political to scientific -based 
controversies, such as whether the government should allow stem cell research or animal testing. 
As part of the WG program, teachers in the four main content areas - English language arts, 
social studies, science and math - present materials related to a controversial issue and explore 
academic language that is embedded in the curriculum through discourse and writing (Snow et 
ah, 2009). For fifteen minutes a day, teachers and students engage in vocabulary instruction 
and/or classroom discussion 

Research Design:. 

As part of a two-year data collection effort conducted from Fall 2010 to Spring 2012, thirteen 
middle schools in an urban school district located in a West Coast metropolitan area of the 
United States participated in a randomized study of the Word Generation intervention. Before 
randomization occurred, state accountability data was used to rank schools on a number of 
school-level variables based on: ethnic minorities, low-income status, English language learners, 
and prior academic achievement. Propensity score matching was used to create a composite 
score from the school-level variables as a means of determining school rankings. To strengthen 
comparability of treatment and control school, each school was paired with another school based 
on closely matching composite scores. The school pairs were then randomized, which resulted in 
one more school assigned to the treatment condition ( n = 7) in comparison with control condition 
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(i n = 6). This study examines the second academic year (2011-2012) of the randomized trial, 
where all but one of the schools in the treatment group participated in our data collection. In turn, 
our sample consists of an equal number of schools randomly assigned to treatment ( n = 6) and 
control conditions (n = 6). 


Data Collection and Analysis: 

Data Collection 

Survey items were developed from a nationally recognized survey of civic measures designed for 
young people aged 12-18 years old that contains reliable psychometric properties on a broad 
range of civic measures including civic behavior, political efficacy, political conversation, and 
news media consumption (Flanagan, Syvertsen, & Stout, 2007). The survey items were also 
modified based on our results from cognitive lab interviews, where adolescents were asked to 
assess their understanding of civic engagement. We also piloted the survey items in a sample of 
seventh graders enrolled in the WG program and found reasonably high alpha reliabilities in the 
civic engagement measures (a =0.74- 0.81). The main dependent variables of interest are the 
following survey items: 

Helping the poor and neighborhood. Students responded to the item, “How often do you 
help poor people in your city?” and answered on a five-point Likert scale of (1) never, (2) 
rarely, (3) sometimes, (4) often, and (5) always. 

Helping the school. Students responded to the item, “How often do you help out at your 
school?” and answered to a five-point Likert scale of (1) never, (2) rarely, (3) sometimes, 
(4) often, and (5) always. 

Helping friends. Students responded to the item, “How often do you help your friends?” 
and answered to a five-point Likert scale of (1) never, (2) rarely, (3) sometimes, (4) often, 
and (5) always. 

These items were used to develop the civic engagement scale and have a reasonably good 
reliability (alpha = 0.60). 

The school district also provided demographic data based on students’ racial-ethnicity identity, 
gender, year in school, school status and academic ability. 

Analysis 

Multiple regression analyses were used to estimate the degree to which students’ participation in 
the WG program is related to their self-reported civic engagement, while including relevant 
controls in the models. In the equation below: 

CIVIC ENG A GEMENT ,= p 0 + [1/ TREA T, + (L FEMALE ) 

+ (L GRADE f + $ 4 GRADE Ji + p 5 ASIAN t 
+ [L BLACKi + p 7 LATINO: + [k OTHER RA CEj 
+ p 9 FREE AND RED U CED L UNCH_STA TUSi 
+ p 0 ENGLISH LANGUAGE LEARNERj 
+ p// SPECIAL EDUCATION + p /2 GATE, 

+ p/.i POLITICAL EFFICACY: 

+ P ,4 ACADEMIC ABILITYi + s, 
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Findings / Results: 

<INSERT TABLE 2> 

Our multiple regression results provide support for the primary research question- participation 
in the Word Generation program has a significant impact on students’ civic engagement, after 
accounting for other covariates {TREAT = 0.069 , p < 0.001). The non-significance in the 
interaction terms also indicates that treatment effects are equal across Asian and Hispanics. 

These results support past research that explain the link between classroom discussion and civic 
engagement - students with opportunities to discuss social and political issues develop civic 
identity or the self-concept of envisioning oneself as an active participant in civic affairs 
(Youniss, McLellan, & Yates, 1997). Another possibility is that adolescents may be stimulated 
by classroom peer effects, where teachers serve as models of democratic and discursive behavior 
that can “pull” previously disengaged classmates to become more civically engaged (Campbell, 
2008,/;. 451). 

Conclusions: 

This study has important implications on policy and practice that are significant on two 
main fronts: (1) supporting discussions of controversial issues can help adolescents develop 
interest in serving the greater community and (2) addressing civic engagement gaps between 
ethnic minorities and mainstream students. Although this study has a number of important 
strengths that include the randomized design and comparison across multiple schools, there are 
several limitations to acknowledge. This study is limited by not having pre-test data, which 
allows for a more precise estimate of the treatment effect. Another limitation is that the analysis 
relies on student-reported assessments of civic engagement, which does not accurately reflect 
students’ actual performance. 

We plan to use Structural Equation Modeling (SEM) to test a communication mediation 
model of youth socialization, in which program effects may be indirectly working through 
informational use of news media and home discussions of politics (Lee, Shah, & McLeod, 2012). 
Also, we can test for treatment and follow-up effects by comparing sixth graders who received 
one year with seventh/eighth graders who received two years of the program. 
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Figure 1 . The Structure-Stimulus-Orientation-Response model (S-S-O-R) model 
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Table 1 

Demographics of the Analytical Sample by Treatment Condition 



Control 

Schools 

Word 

Generation 

Schools 

All 

Schools 

n 

% 

n 

% 

N 

% 

Female 

968 

50.0% 

1,595 

49.5% 

2,563 

49.7% 

Special Education 

175 

9.0% 

261 

8.1% 

436 

8.5% 

Free and Reduced Lunch Eligible 

1,344 

69.5% 

1,944 

60.4% 

3,288 

63.8% 

English Language Learner 

1,219 

63.0% 

1,710 

53.1% 

2,929 

56.8% 

Asian 

1,115 

57.6% 

1,673 

51.9% 

2,788 

54.1% 

African-American 

146 

7.6% 

209 

6.5% 

355 

6.9% 

Flispanic 

397 

20.5% 

597 

18.5% 

994 

19.3% 

White 

110 

5.6% 

332 

10.3% 

442 

8.6% 

Other 

60 

3.1% 

160 

4.9% 

220 

4.3% 

Total Students 

2,352 

40.1% 

3,518 

59.9% 

5,870 

100.0% 
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Table 2 

Regression Models Predicting Students’ Self-Reported Civic Engagement Controlling for 
Students' Individual Characteristics with Standard Errors in Parentheses 



Without 

Covariates 

With 

Covariates 

Interactions 

Treatment (n = 4,072) 

0.065*“ 

0.069*** 

0.083* 


(0.018) 

(0.019) 

(0.038) 

Female ( n = 3,217) 


0.249*** 

(0.018) 

0.249*" 

(0.018) 

Grade 7 (n = 2,444 ) 


0.016 

(0.023) 

0.016 

(0.023) 

Grade 8 (n = 2,512) 


-0.030 

(0.022) 

-0.030 

(0.023) 

FRL (n = 4,241) 


-0.034 

(0.021) 

-0.033 

(0.021) 

GATE (72 = 3,425) 


-0.042 

(0.022) 

-0.042 

(0.022) 

ELL (72 = 3,729) 


-0.025 

(0.022) 

-0.025 

(0.022) 

Special Education (n = 625) 


-0.056 

(0.054) 

-0.056 

(0.054) 

Asian ( 72 = 3,457) 


0.055 

(0.028) 

0.061 

(0.041) 

Hispanic (72 = 1,349) 


-0.0701* 

(0.034) 

-0.040 

(0.049) 

Black (72 = 490) 


0.082 

(0.045) 

0.084 

(0.045) 

Other Ethnicity (72 = 279) 


0.129“ 

(0.050) 

0.129“ 

(0.050) 

Political Interest 
Asian x Treatment 
Hispanic x Treatment 


0.242*** 

(0.009) 

0.242*“ 

(0.009) 

-0.008 

(0.045) 

-0.049 

(0.057) 

Intercept 

3.084*“ 

2.323*** 

2.312*" 


-0.014 

-0.059 

-0.063 

N 

6,244 

4,968 

4,968 

R 2 

0.002 

0.176 

0.175 


Note. Standard errors in parentheses; Civic engagement is an aggreggate measure relating to students' reported frequency of 
helping their school, friend and poor people in their city on a Likert scale of 1 "rarely" to 5 "always"; ELL- Refers to English 
Language Learner status; FRL refers to free and reduced lunch status; GATE refers to District Gifted and Talented Education 
status; * p < 0.05, ** p < 0.01, *** p < 0.00. 
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